Three dimensional depth mapping using dynamic structured light

ABSTRACT

Apparatus for generating a dynamic structured light pattern for optical tracking in three-dimensional space, comprises an array of lasers, such as a VCSEL laser array, to project light in a pattern into a three-dimensional space; and an optical element or elements arranged in cells. The cells are aligned with subsets of the laser array, and each cell individually applies a modulation, in particular an intensity modulation, to light from the laser or lasers of the subset, to provide a distinguishable and separately controllable part of the dynamic structured light pattern. A method of generating a structured light pattern is disclosed, in which light is provided from an array of lasers, and light is individually projected from subsets of the array of lasers to provide differentiated parts of the structured light pattern.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/859,871, filed Apr. 27, 2020, which is a continuation of U.S.application Ser. No. 16/032,304, filed Jul. 11, 2018, now U.S. Pat. No.10,687,047, which is a continuation of U.S. application Ser. No.15/030,851, filed Apr. 20, 2016, now U.S. Pat. No. 10,091,494, which isa National Phase Application of International Application No.PCT/IL2014/050922, filed Oct. 23, 2014, which claims the benefit of U.S.Application No. 61/894,471, filed Oct. 23, 2013, all of which are hereinincorporated by reference in their entirety.

BACKGROUND

The present invention relates to three-dimensional depth mapping usingstructured light, and more particularly, but not exclusively, to asystem for tracking in order to provide input to a computer or likedevice.

Tracking in order to provide input to a computer, or other computationaldevice, involves such ubiquitous technologies as the computer mouse.Tracking of styluses and fingers in a three-dimensional field in frontof the computer is also available and uses various tracking technologiessuch as visual and IR imaging and ultrasonic. The term ‘tracking’ mayrefer to following the positioning and motion of an object, and includesprocessing of inputs received at the tracking computer in order todetermine the position or motion. Thus in the case of a computer mouse,tracking may include processing the mouse outputs to determine motion.In the case of an object being followed visually, the term tracking mayinclude image processing of successive frames capturing the object.

One method of imaging simply uses cameras to view and process a scene.The cameras may follow specific marks that are placed in the scene orthe imaging system can look for specifically recognizable features suchas fingers.

Drawbacks of such visual imaging include a requirement that thethree-dimensional area is sufficiently lit up. Furthermore the onlyfeatures that can be tracked are features that are recognized inadvance, and motion tracking combined with feature recognition may notgive accurate results.

In order to overcome the above problems, tracking using structured lightwas introduced. A known pattern of pixels is projected onto the spacewhere tracking is to occur. The way that the pattern deforms on strikingsurfaces allows the vision system to calculate the depth and surfaceinformation of objects in the scene. Typical patterns used are grids orhorizontal bars. Various devices use structured light patterns to enablethe use of gesture recognition and 3D depth mapping. The structuredlight patter transmitter includes a laser emitter and a diffractiveoptical element (DOE).

Projecting a narrow band of light onto a three-dimensionally shapedsurface produces a line of illumination that appears distorted fromother perspectives than that of the projector, and can be used for anexact geometric reconstruction of the surface shape. A faster and moreversatile method is the projection of patterns consisting of manystripes at once, or of arbitrary fringes, as this allows for theacquisition of a multitude of samples simultaneously. Seen fromdifferent viewpoints, the pattern appears geometrically distorted due tothe surface shape of the object.

Although many other variants of structured light projection arepossible, patterns of parallel stripes are widely used. The displacementof the stripes allows for an exact retrieval of the 3D coordinates ofany details on the object's surface.

One known method of stripe pattern generation is the laser interferencemethod, which utilizes two wide planar laser beam fronts. Interferencebetween the beam fronts results in regular, equidistant line patterns.Different pattern sizes can be obtained by changing the angle betweenthese beams. The method allows for the exact and easy generation of veryfine patterns with unlimited depth of field. Disadvantages are high costof implementation, difficulties providing the ideal beam geometry, andlaser typical effects such as speckle noise and the possibleself-interference with beam parts reflected from objects. Furthermore,there is no means of modulating individual stripes, such as with Graycodes.

Specifically, a disadvantage of using a single source emitter such as anedge emitter laser diode is the fact that the light pattern that itproduces can be controlled only as a single unit. This means that thelight pattern can be entirely turned on, off or dimmed but cannot bechanged dynamically.

Structured light patterns may be constructed using invisible light suchas IR light.

Alternatively high frame rates may render the structured lightimperceptible to users or avoid interfering with other visual tasks ofthe computer.

The vertical-cavity surface-emitting laser, or VCSEL is a type ofsemiconductor laser diode in which laser beam emission is perpendicularfrom the top surface, as opposed to conventional edge-emittingsemiconductor lasers which emit from surfaces formed by cleaving theindividual chip out of a wafer.

There are several advantages to producing VCSELs, as opposed toedge-emitting lasers. Edge-emitters cannot be tested until the end ofthe production process. If the edge-emitter does not function properly,whether due to bad contacts or poor material growth quality, theproduction time and the processing materials have been wasted. VCSELscan be tested at several stages throughout the process to check formaterial quality and processing issues. For instance, if the vias havenot been completely cleared of dielectric material during the etch, aninterim testing process may be used to determine that the top metallayer is not making contact with the initial metal layer. Additionally,because VCSELs emit the beam perpendicularly to the active region of thelaser, tens of thousands of VCSELs can be processed simultaneously on athree inch Gallium Arsenide wafer. Furthermore, even though the VCSELproduction process is more labor and material intensive, the yield canbe controlled to a more predictable outcome.

There is a significant advantage in that the use of VCSEL laser arrayfor a structured light system, in that use of the array allows for areduction in the size of the structured light transmitter device. Thereduction is especially important for embedding the transmitter indevices with size restrictions such as a mobile phone or wearabledevices.

However, despite the above advantages, the VCSEL array is not currentlyused for structured light scanning systems for a number of reasons. Manydiffraction patterns require a coherent Gaussian shaped beam in order tocreate the high density patterns needed for high resolution tracking.The VCSEL array merely provides multiple individual Gaussian beamspositioned next to each other and usually with overlap between them. Themultiple points and overlap between them reduce the detectionperformance in high density areas in the light pattern and restrict theuse of various diffractive design techniques that requires a pre-definedGaussian beam. Such designs include a Top-Hat design, Homogeneous linegenerators, and other complex high performance structures.

Indeed the problem with a standard diffractive design is that the entireVCSEL laser array is used as a single light source. Thus, when using amultiple spot design the array image is obtained for each spot insteadof having a focused Gaussian beam. A diffractive design that requires aGaussian beam as an input will not get the required output at all. Theproblem becomes more severe in dense light patterns, because in theselight patterns there is a need to focus the source beam onto a tiny spotin order to separate the features and this is not possible if the lightsource is an array of lasers.

SUMMARY

The present embodiments provide an array of VCSEL lasers, where thelasers of the array are modulated individually or in groups. Theindividual lasers or groups may be modulated statically or dynamicallyto provide and alter a structured light pattern as needed.

Thus each laser in the array, or group of lasers being moderatedtogether, is provided with its own optical element, typically adiffraction element. The diffraction element can be individuallycontrolled so that the overall structured light pattern can be selectedfor given circumstances and/or can dynamically follow regions ofinterest.

According to a first aspect of the present invention there is providedapparatus for generating a structured light pattern, comprising:

an array of lasers arranged to project light in a pattern into athree-dimensional space; and a plurality of optical elements, respectiveoptical elements defining cells, the cells being aligned with respectivesubsets of the array of lasers, the optical element of each cellindividually applying a modulation to light passing through the opticalelement from the respective subset to provide a distinguishable part ofthe structured light pattern.

In an embodiment, the optical modulation is any of a diffractivemodulation, a refractive modulation, and a combination of a diffractiveand a refractive modulation.

In an embodiment, the optical elements and the subset of the array oflasers comprising a respective cell are constructed from a single moldedelement.

In an embodiment, a width of the cell is 1 mm or less.

In an embodiment, a width of the optical element is 1 mm or less.

In an embodiment, the cells are individually controllable to change thediffractive modulation.

In an embodiment, the cells are controllable dynamically to providechanges to the structured light pattern based on receiving and analyzingat least one captured frame, the frame comprising a plurality of pixelsin a two-dimensional layout.

In an embodiment, the cells are further controllable in respect ofpositioning within the structured light pattern and in respect of ashape applied to the light.

In an embodiment, the dynamic control is configurable to apply increasedresolution of the structured light pattern to parts of the scene toapply reduced resolution of the structured light pattern to other partsof the scene.

In an embodiment, the dynamic changes to the structured light patterncomprise changes to orientation of the structured light pattern.

In an embodiment, the dynamic changes to the structured light patterncomprise cell wise change.

In an embodiment, the change is any of a change in intensity, a changein polarization, a change in filtering parameters, and a change infocus.

In an embodiment, the subsets are any of individual lasers, pairs oflasers, triplets of lasers, combinations of different sizes of lasers,and dynamically changing combinations of lasers.

In an embodiment, light projected from respective subsets is tiled oroverlapped.

In an embodiment, the laser array comprises a VCSEL laser array.

In an embodiment, the laser array comprises a laser bar.

The apparatus may be incorporated into any of: a computer, a laptopcomputer, a mobile communication device, a tablet device, a gameconsole, and a movement capture device.

The apparatus may be used for tracking of a three-dimensional scene.

According to a second aspect of the present invention there is provideda method of generating a structured light pattern for three-dimensionaltracking, the method comprising: Providing light from an array oflasers; and individually projecting light from subsets of the array oflasers to provide differentiated parts of the structured light pattern.

According to a third aspect of the presenting invention there isprovided apparatus for generating a structured light pattern,comprising: an array of lasers arranged to project light in a patterninto a three-dimensional space; and a plurality of optical elementcells, the cells being aligned with respective subsets of the array oflasers, each cell individually applying an intensity modulation to lightpassing through the element from the respective subset to provide adistinguishable part of the structured light pattern.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods, and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

Implementation of the method and/or system of embodiments of theinvention can involve performing or completing selected tasks manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of embodiments of the method and/or systemof the invention, several selected tasks could be implemented byhardware, by software or by firmware or by a combination thereof usingan operating system.

For example, hardware for performing selected tasks according toembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to embodiments of theinvention could be implemented as a plurality of software instructionsbeing executed by a computer using any suitable operating system. In anexemplary embodiment of the invention, one or more tasks according toexemplary embodiments of method and/or system as described herein areperformed by a data processor, such as a computing platform forexecuting a plurality of instructions.

Optionally, the data processor includes a volatile memory for storinginstructions and/or data and/or a non-volatile storage, for example, amagnetic hard-disk and/or removable media, for storing instructionsand/or data. Optionally, a network connection is provided as well. Adisplay and/or a user input device such as a keyboard or mouse areoptionally provided as well.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a simplified schematic diagram illustrating a basic setup of astructured light generating apparatus according to an embodiment of thepresent invention, wherein a cell of the optical element, having anindividual pattern, corresponds to a specific VCSEL in the VCSEL laserarray;

FIG. 2 is a simplified schematic diagram of the embodiment of FIG. 1 andshowing a light pattern construction wherein each cell creates a partialtile of an overall structured light pattern;

FIG. 3 is a schematic diagram illustrating a variation of the embodimentof FIG. 1, wherein each cell consists of a triplet of VCSEL lasers;

FIG. 4 is a schematic diagram illustrating a variation of the embodimentof FIG. 1 in which different light patterns are created by differentcell patterns as before but the cells are of various sizes andorientations; and

FIG. 5 is a schematic diagram illustrating a variation of the embodimentof FIG. 1 in which different light patterns are created by differentcell patterns as before and individual cells are responsible for variouslight features of the overall structured pattern that are notnecessarily organized in separated tile structures;

FIG. 6 is a schematic diagram illustrating a tracking system with acamera and a processor according to embodiments of the presentinvention;

FIG. 7 is a schematic diagram illustrating different modulations thatcan be applied by the optical element according to embodiments of thepresent invention;

FIG. 8 is a schematic diagram illustrating placement or redirecting of abeam by an optical element in accordance with embodiments of the presentinvention;

FIG. 9 is a simplified diagram illustrating the use of an opticalelement of the present invention for focusing of a beam;

FIG. 10 is a simplified diagram showing an optical element carrying outbeam shaping in accordance with embodiment of the present invention;

FIG. 11 is a simplified diagram showing a multistage optical elementcomprising focusing, beam directing and beam shaping, according to anembodiment of the present invention;

FIGS. 12a and 12b illustrate the effect of using stripes on a pointingfinger and show how the stripes may need to be reoriented if they areparallel to the finger;

FIG. 13 illustrates dynamic changing of patterns according toembodiments of the present invention, in this case between vertical andhorizontal lines;

FIG. 14 illustrates dynamic changing of patterns from a large to a smallfield of view, according to embodiments of the present invention;

FIG. 15 illustrates dynamic changing of patterns between low density andhigh density lines according to embodiments of the present invention;

FIG. 16 illustrates dynamic changing of patterns between continuouslines and lines made up of dashes, according to embodiments of thepresent invention;

FIG. 17 illustrates dynamic changing of patterns between lines of lowintensity and lines of high intensity according to embodiments of thepresent invention;

FIG. 18 illustrates changing of light intensity depending on thedarkness or brightness of an object according to embodiments of thepresent invention; and

FIG. 19 is a simplified flow chart illustrating dynamic changing ofpatterns carried out following frame analysis according to embodimentsof the present invention.

DETAILED DESCRIPTION

The present invention, in some embodiments thereof, relates tothree-dimensional depth mapping using structured light, and moreparticularly, but not exclusively, to a system for tracking in order toprovide input to a computer.

The following applications to the same assignee are hereby incorporatedby reference as if fully set forth herein, namely U.S. patentapplication Ser. No. 13/497,589, filed Sep. 19, 2010, InternationalPatent Application No. W02013/088442 filed 13 Sep. 2012, U.S.Provisional Patent Application No. 61/926,476 filed 13 Jan. 2014, andU.S. Provisional Patent Application No. 62/035,442 filed 10 Aug. 2014.

As discussed above, various devices use structured light patterns toenable gesture recognition and 3D depth mapping. A structured lightpattern transmitter includes a light source, for example a laseremitter, and an optical element such as a diffractive optical element(DOE). As many diffractive designs requires a coherent Gaussian shapedbeam in order to create high density patterns, the use of a VCSEL laserarray is generally not possible. The optical element, for example theVCSEL array, creates multiple Gaussian shaped beams with overlap, whichreduces the detection performance in high density areas in the lightpattern and restricts the use of various diffractive design techniquesthat require a pre-defined Gaussian beam. Such designs include a Top-Hatdesign, Homogeneous line generators, and other complex high performancestructures.

There is, however, a significant advantage in the use of VCSEL laserarray to reduce the size of the structured light transmitter device.This is especially important for embedding the transmitter in deviceswith size restrictions such as a mobile phone or wearable devices.

Thus, the present embodiments provide an array of VCSEL lasers, wherethe lasers of the array are modulated individually or in groups. Theindividual lasers or groups may be modulated statically or dynamicallyto provide and alter a structured light pattern as needed.

Each laser in the array, or group of lasers being moderated together, isprovided with its own cell of an optical element, typically adiffraction element. The cells of the diffraction element can beindividually controlled to provide different light patterns at differentparts of the array so that the overall structured light pattern can beselected for given circumstances and/or can dynamically follow regionsof interest, as will be discussed in greater detail below. Typicalstructures include stripes, grids, and dots.

As mentioned, a problem using a single source emitter such as an edgeemitter laser diode is the fact that the light pattern that it producescan be controlled only as a single unit. Consequently, the light patterncan be entirely turned on, off or dimmed but cannot be changeddynamically. By contrast, each VCSEL laser in the array according to thepresent embodiments can be controlled individually, since control is atthe level of the cell of the optical element, and suitable design of,for example, the DOEs may provide a dynamic light pattern that canproduce flexible detection for various use cases and feedbacks. In thefollowing the term ‘cell’ relates to a surface operable with a singlelaser or any group of lasers that are operated together to provide aparticular part of the pattern. The cell structure may changedynamically as the pattern is changed.

Instead of a diffractive optical element, a refractive element may beused, or a combination of diffractive and refractive elements.

According to one embodiment the cell and/or the optical element of thecell may be limited by size, for example, the cells may be of the orderof magnitude of less than 1 mm.

In more detail, the present embodiments relate to a generic lens/DOEdesign that enables the use of a VCSEL array to produce a dynamic lightpattern. The DOE is positioned on the surface adjacent to the VCSELarray such that the plane of the DOE is parallel to the plane of thearray/matrix. In the present embodiments, the surface of the DOE isdivided into cells. Each cell represents an area which is positionedabove a single VCSEL laser or a sub-group of VCSELs that are intended tobe controlled together. For clarity, the lasers in the group or subgroupare controlled together, separately from lasers in other groups orsubgroups.

A unique diffractive pattern may be designed for each cell, creatingpart of the required structured light pattern. The individual patterngenerated by the VCSEL lasers following each cell's diffractive patterncreates a sub pattern of the structured light. The overall pattern isthen formed from the patterns of the individual cells, for example bytiling, overlapping, or other ways for positioning of individualfeatures.

The Design of each cell may comprise two optical functions. A firstpositioning function determines the position of the light feature in theentire structured light image.

For example, such a positioning function may consist of a prism blazedgrating bending the position of the diffracted light to the actualposition of the tile in the required pattern. A second, optical,function relates to the shape of the light feature. Examples of suchoptical functions may include a line generator, a multi spot pattern orother features or sub-features of the light pattern.

With suitable alignment between the VCSEL laser matrix and the cellbased DOE any pattern can be achievable since the adjacent-Gaussiansbeam shape of the entire array is avoided as a single light sourceperspective.

In another embodiment, a dynamic light pattern is presented. Each cellcan be controlled individually in terms of output intensity by applyingdifferent currents to the DOE at the appropriate location, and thusvarious features in the structured light pattern can be controlled aswell.

Dynamic control, meaning changing the cell pattern during the course oftracking, enables various functions. That is to say, the optical elementof each cell may be dynamically changed according to received data, forexample in a sequence beginning with an initial configuration of thelasers. A frame is captured of the scene, the frame for example being atwo-dimensional array of pixels. The received frame is analyzed. Then anew laser configuration is reached based on the analyzed frame. The newlaser configuration then becomes the initial configuration for the nextstage as the cycle continues. An example is illustrated and discussedbelow with respect to FIG. 19. For example it is possible to confirm thesource of each viewed light feature by altering the intensity of asuspected VCSEL. Such confirmation is very useful in triangulation baseddepth detection methods. A further function is the ability to alter thedensity of the light pattern. Altering the density enables higherresolution sampling after the scenario has been roughly mapped, and maycorrect intensity errors caused by the camera setup or other scenarioeffects such as lighting conditions, reflectivity etc. Furthermore, itis possible to change the orientation of features in the light pattern.

Altering either the intensity or the orientation provide ways of givingthe image processing software an additional chance to process the scenefrom what is effectively a new perspective, and according to datareceived from the camera, as discussed above.

Reference is now made to FIG. 1, which is a simplified schematic diagramthat describes a basic arrangement of a device for three dimensionaltracking using patterned light according to the present embodiments. Itis to be appreciated that in this and the following figures, dotted anddashed lines represent conceptual and structural lines as provided bythe context, and not necessarily physical lines visible in therespective embodiment.

In the arrangement of FIG. 1, a light producing array, for example alaser array, which may be VCSEL array 10, comprises a matrix of lasers12.1.1 . . . 12.n.n. An optical element 14 comprises cells 16.1.1 . . .16.n.n, which are aligned externally to the lasers, so that individualcells are located to modulate the light from a respective laser. Eachlaser has a separate cell of for example the diffractive optical elementand each separate cell has a different diffractive pattern. Zoom 18 ofthe cells 16 shows four diffractive elements each with a uniquediffractive pattern. Thus the diffractive optical element has multipleindividual designs, one per cell, where each design diffracts the lightof one VCSEL laser.

The setup may generate a structured light pattern from the array oflasers which is projected into a three-dimensional space for trackingobjects and parts of scenes within that space. The structured lightpattern may be any suitable pattern that can be parsed to provide depthinformation to the computer, and includes patterns including regions ofstripes, grids, and/or dots.

The cells are aligned with subsets of the array of lasers, and each cellindividually applies a diffractive modulation to light passing through,so that each subset provides a distinguishable part of the structuredlight pattern.

The cells 16.1.1 . . . 16.n.n may be individually controllable to changethe diffractive modulation. Thus different parts of the pattern may bedifferent, and different structured light patterns can be used indifferent circumstances, or in different parts of the scene.

The cells may further be controlled dynamically to provide changes tothe structured light pattern. Thus, the pattern may change to increaseresolution in parts of the scene deemed to be of interest and/or mayreduce resolution in parts of said scene deemed not to be of interest.Alternatively, particular parts of the pattern may be momentarilychanged to indicate a particular light source reaching a particular partof the scene, so as to give additional clues for triangulation and depthestimation. Typically the intensity would be changed. That is to say thechange is based on controlling the intensity of the array of lasersaffecting the cell. As alternatives, the polarization, filteringparameters, or focal length may be changed or any other feature of thelight apparent to those skilled in the art.

The intensity may be changed over part or all of the pattern. Forexample, parts of the scene may be brightly lit by incident light andother parts of the scene may be dimly lit. High intensity light may beaimed at the brightly lit parts and low intensity light to the dimly litparts, thus saving power.

Alternatively, the density of the pattern may be changed or theorientation of the pattern may be changed, typically to give a differentview of the scene for the tracking and depth estimation, as will bediscussed in greater detail below. Regarding orientation, a feature ofthe scene may be more effective illuminated in a given orientation. Forexample a long narrow feature may be most effectively illuminated bystripes perpendicular to its longitudinal direction. The stripedirection may be updated as the orientation of the feature changes overtime. Density too may be altered over time to allow particular featuresto be tracked more accurately, or as fine features come into view.

The subsets shown in FIG. 1 are individual lasers. However, as will bediscussed in connection with the following figures, alternative subsetsinclude pairs of lasers, triplets of lasers, combinations of differentsizes of laser groupings, and dynamically changing combinations oflasers. The cells may be of the order of magnitude of 1 mm.

The projected light may be organized as tiles or overlappings or anyother suitable arrangement.

The structured pattern based tracking arrangement may be incorporatedinto a computer, including a laptop computer, or a tablet or pod deviceor a mobile communication device such as a mobile telephone, or a gameconsole, or a movement capture device, such as the kind of device usedby animators to capture movements by actors, or any kind of device wheretracking in three dimensions may provide a useful input.

Instead of a single diffractive optical element arranged in cells,multiple diffractive elements may be used, and all reference to cellsherein are to be construed as additionally referring to separate opticalelements.

The present embodiments thus allow generating of a structured lightpattern using a VCSEL laser array and a diffractive optical element(DOE) and controlling each VCSEL individually for altering thestructured light pattern dynamically.

Dynamic control may include altering the intensity of individualfeatures in the light pattern, or the density or orientation of thefeatures, or indeed turning on and off individual features of the lightpattern, or changing the any of the above based on feedback from thestructured light analysis. For example if the analysis reveals thatgreater resolution is needed, then the density may be increased, as willbe discussed in greater detail below. If the analysis reveals thatexternal lighting is interfering with the readout, then intensity isincreased. If light reflections from external lighting is a problem thenthe orientation of the light pattern may be changed. As mentioned above,aside from issues of external lighting, it is possible to reorient thestripes so as to keep them perpendicular to the object being tracked.Thus feedback from the scenario is used to alter the lighting pattern.Incident lighting conditions in the scene can be dealt with by adjustingbrightness over affected parts of the pattern. Thus, control ofindividual cells can allow certain areas of the pattern to be modifiedbased on feedback from the scene and not others. The pattern may bedynamically changed between grids, stripes and dots or any other patternthat may be used, and different cells may have different patterns, thecells being dynamically redefined as needed.

Reference is now made to FIG. 2, which is a simplified schematic diagramshowing one way in which the arrangement of FIG. 1 may be used. In FIG.2, the array of lasers is indicated by numeral 20 and each laser 22.1.1. . . 22.n.n corresponds to a single cell. The cell illuminates adifferent tile 24.1.1 . . . 24.n.n respectively in a forward projection26. In projection 26, each tile has a different pattern, and all thetiles merge together to form a complete light pattern.

Reference is now made to FIG. 3, which is a simplified schematic diagramillustrating a variation of the arrangement of FIG. 1 in which a cellcaters for two or more VCSEL lasers, specifically three in the caseillustrated. In FIG. 3, the array of lasers is indicated by numeral 30and individual lasers are combined by the optical element into athree-laser cell 32.1.-32.3. The cells illuminate different tiletriplets 34.1 . . . 34.n respectively in a forward projection 36. Inprojection 36, all tiles in the same triplet share a pattern but eachtriplet has a different pattern within the overall projected lightpattern 36.

Reference is now FIG. 4 which is a simplified schematic diagramillustrating a variation of the arrangement of FIG. 1 in which differentcells have different designs and different orientations. In FIG. 4, thearray of lasers is indicated by numeral 40 and individual lasers 42.1.1. . . 42.n.n form one and two laser cells. The cells illuminatedifferent tiles 44.1.1 . . . 44.n.n respectively in a forward projection46. In projection 46, the tiles are of variable sizes, and there is onecell size of two sharing a single pattern.

Reference is now made to FIG. 5, which is a simplified schematic diagramillustrating a further variation of the arrangement of FIG. 1 in whicheach cell is responsible for producing various light features of thepattern that are not necessarily organized in separate tile structures.In FIG. 5, the array of lasers is indicated by numeral 50 and individuallasers 52.1.1 . . . 52.n.n form individual one-laser cells. The laser52.1.1 illuminates tile triplet 54.1 in a forward projection 56 toprovide horizontal lines over the triplet of tiles.

Reference is now made to FIG. 6 which is a simplified schematic diagramwhich illustrates an exemplary system for tracking according toembodiments of the present invention. VCSEL array 60 produces laserlight which is modulated by optical element 62 to form a pattern shoneinto a 3D volume 64. Camera 66 monitors the 3D volume and produces imageframes which are analyzed by processor 68. The processor 68 both carriesout tracking and also modifies the light modulations in order to improvetracking, as will be discussed in greater detail below. The processormay be an external processor such as that of a mobile device. The cameramay a digital camera or any imagery sensor which may be based, forexample on a complementary metal oxide silicon (CMOS) array. Theprocessor may be connected to both the camera and the lasers, so thatonce the frame is analyzed the processor controls the lasersaccordingly.

Reference is now made to FIG. 7, which is a simplified schematic diagramillustrating various operations, that is different optical functions ofthe optical element, available for modulating the laser light from asingle or sub optical cell 70, or a number of cells such as 18 shown inFIG. 1, using laser array 72 to form the projected patterns. Anyparticular cell may allow the position of the beam to be modified, 76,or the phase of the beam may be modified 78, or the focus may bemodified, 80, or the shape of the beam may be modified 82, or theintensity of the beam may be modified 84, or the polarization of thebeam may be modified 86. The above is not an exhaustive list and othermodifications will be apparent to those skilled in the art. Additionallythe optical functions may be utilized by a single optical element or bymultiple optical elements as shown in FIG. 11, wherein multiplefunctions such as focus, shape etc. are incorporated into the cellelement.

Reference is now made to FIG. 8, which shows an optical element 90changing a direction of a light beam. Features 92 emerging from the bodyof the device, because the change in direction of the light beam, anddepending on the construction may be due to refraction, diffraction or acombination of refraction and diffraction. In the example illustrated, asaw tooth configuration in which tooth-like shapes have a downwardlysloping upper face and a horizontal lower face because downward bendingof the light.

Reference is now made to FIG. 9 which shows a construction 94 forfocusing of the light beam. Features 96 are arranged in a saw toothconfiguration as in FIG. 8, but the orientation of the tooth-likefeatures is exchanged in the lower half of the construction, causing theupper and lower halves of the beam to meet at focal point 98.

Reference is now made to FIG. 10, which is a simplified diagram showinga variation 100 of the optical element for shaping of the beam. A presetrandom function is used to define a surface 102 of the optical elementin order to shape the beam into a way that differentiates the beam fromother beams.

Reference is now made to FIG. 11, which is a simplified diagram showingan optical element 110, which may relate to a cell or a sub cell, andwhich combines the optical elements of the previous three figures. Afirst part 112 focuses a beam produced by VCSEL array 113. Part 114bends the beam, in this case in a downward direction, and part 116shapes the beam.

Reference is now made to FIG. 12A which shows a hand 120 being trackedby a pattern 122 of horizontal stripes. It is apparent that the patternhappens to coincide with the lengths of the fingers and thusinformation/data provided regarding the fingers is limited and it isdifficult to identify the shape of the object. Nevertheless the fingersmay often be the points of major interest as they provide the gesturesthat the system uses as commands. The system may therefore change, forexample automatically the orientation of the stripes to that shown inFIG. 12B so that the stripes cross the fingers, hence providing moreinformation. The change is done based on image analysis feedback asincluded in the flow chart.

FIGS. 13 to 17 illustrate various changes that may be made to the lightpattern in order to improve tracking. FIG. 13 shows a change in theorientation of the stripes from horizontal to vertical as in theprevious figure. FIG. 14 shows a narrowing of the field of view.Narrowing may be useful for example in a case where fingers are afeature of interest. Once the fingers are found then the field of viewmay be narrowed to concentrate on them.

FIG. 15 shows an increase in density of the stripes. FIG. 16 shows achange in shape from pure stripes to lines of dashes. FIG. 17 showschanges in intensity. Lines of low intensity in the first image arechanged into lines of high intensity in the second image and vice versa.Suitable selection of the low and high density lines may be used totrack an object in the depth dimension.

Reference is now made to FIG. 18, which shows how light and dark objectsmay be tracked using the embodiment of FIG. 2. Parts that are the sameas in FIG. 2 are given the same reference numerals and are not describedagain except as needed for an understanding of the present embodiment.In FIG. 18, a light colored object 180 is found in the cell area 24.11.The intensity of the stripes is decreased. A dark colored object 182, bycontrast, is found in the area of cell 24.2.3. For the dark object theintensity is increased. The different cells are able to operateindependently and each area reacts appropriately to the objects it isrespectively tracking.

Reference is now made to FIG. 19, which is a simplified flow chartillustrating a procedure for modifying the pattern in a cell accordingto an embodiment of the present invention. A particular cell starts acycle with an initial configuration—190. A frame is captured 192 andanalyzed 194. From the analysis a new laser configuration is determined196 based on defined analysis guidelines 198. That is to say for anyparticular situation there is a defined change in the pattern, whichdefined change may obtain more information about the object in theframe. The new laser configuration then serves as the initial laserconfiguration for the next frame.

The terms “comprises,” “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”.

The term “consisting of’ means “including and limited to.”

As used herein, the singular form “a,” “an” and “the” include pluralreferences unless the context clearly dictates otherwise.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment, and the abovedescription is to be construed as if this combination were explicitlywritten. Conversely, various features of the invention, which are, forbrevity, described in the context of a single embodiment, may also beprovided separately or in any suitable sub combination or as suitable inany other described embodiment of the invention, and the abovedescription is to be construed as if these separate embodiments wereexplicitly written. Certain features described in the context of variousembodiments are not to be considered essential features of thoseembodiments, unless the embodiment is inoperative without thoseelements.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives,modifications, and variations will be apparent to those skilled in theart. Accordingly, it is intended to embrace all such alternatives,modifications, and variations that fall within the spirit and broadscope of the appended claims.

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the extent thatsection headings are used, they should not be construed as necessarilylimiting.

What is claimed is:
 1. An optical cell comprising: a plurality ofoptical elements in optical series and aligned with a correspondinglaser of an array of lasers, the plurality of optical elementsconfigured to apply a modulation to light previously emitted from thecorresponding laser that later passes through the plurality of opticalelements forming a partial tile of a structured light pattern, at leastone of the optical elements configured to project the partial tile ofthe structured light pattern for illuminating a portion of a scene. 2.The optical cell of claim 1, wherein the plurality of optical elementscomprises an optical element configured to change a direction of thelight emitted from the corresponding laser.
 3. The optical cell of claim2, wherein the optical element is configured to change the direction ofthe light by diffracting the light.
 4. The optical cell of claim 2,wherein the optical element is configured to change the direction of thelight by refracting the light.
 5. The optical cell of claim 2, whereinthe optical element comprises features arranged in a saw toothconfiguration that are configured to change the direction of the lightby bending the light.
 6. The optical cell of claim 1, wherein theplurality of optical elements comprises an optical element configured tofocus the light emitted from the corresponding laser.
 7. The opticalcell of claim 6, wherein the optical element comprises features arrangedin a saw tooth configuration that focus the light to a focal point. 8.The optical cell of claim 1, wherein the plurality of optical elementscomprises an optical element configured to shape the light emitted fromthe corresponding laser into a unique shape.
 9. The optical cell ofclaim 8, wherein the unique shape depends on a shape of a surface of theoptical element defined based on a preset random function.
 10. Theoptical cell of claim 1, wherein the modulation is selected from a groupconsisting of: a diffractive modulation, a refractive modulation, and acombination of a diffractive and a refractive modulation.
 11. Theoptical cell of claim 1, wherein the plurality of optical elements inoptical series comprises: a first optical element configured to focusthe light emitted from the corresponding laser; a second optical elementconfigured to generate bending light by changing a direction of thefocused light; and a third optical element configured to shape thebending light to generate the partial tile of the structured lightpattern.
 12. The optical cell of claim 1, wherein a resolution of theportion of the scene corresponding to the partial tile of the structuredlight pattern is modified in accordance with the applied modulation. 13.The optical cell of claim 1, wherein the array of lasers is selectedfrom a group consisting of: individual lasers, pairs of lasers, tripletsof lasers, combinations of different sizes of lasers, and dynamicallychanging combinations of lasers.
 14. The optical cell of claim 1,wherein the array of lasers and the at least one optical element areconfigured to project the partial tile into a three-dimensional space totrack a three-dimension scene.
 15. A method comprising: controlling anoptical cell comprising a plurality of optical elements in opticalseries aligned with a corresponding laser of an array of lasers to applya modulation to light previously emitted from the corresponding laserthat later passes through the plurality of optical elements forming apartial tile of a structured light pattern; and projecting the partialtile of the structured light pattern using at least one of the opticalelements for illuminating a portion of a scene.
 16. The method of claim15, further comprising: changing a direction of the light emitted fromthe corresponding laser using an optical element of the plurality ofoptical elements.
 17. The method of claim 15, further comprising:focusing the light emitted from the corresponding laser to a focal pointusing an optical element of the plurality of optical elements, the focalpoint depending on features of the optical element.
 18. The method ofclaim 15, further comprising: shaping the light emitted from thecorresponding laser into a unique shape using an optical element of theplurality of optical elements, the unique shape depending on a shape ofa surface of the optical element defined based on a preset randomfunction.
 19. The method of claim 15, further comprising: focusing thelight emitted from the corresponding laser; changing a direction of thefocused light to generate bending light; and changing a shape of thebending light to generate the partial tile of the structured lightpattern.
 20. An apparatus comprising: an optical cell comprising aplurality of optical elements in optical series and aligned with acorresponding laser of an array of lasers, the plurality of opticalelements configured to apply a first modulation to light previouslyemitted from the corresponding laser that later passes through theplurality of optical elements forming a partial tile of a structuredlight pattern, at least one of the optical elements configured toproject the partial tile of the structured light pattern forilluminating a portion of a scene; and a processor configured to:determine a second modulation different from the first modulation byanalyzing captured light reflected from the scene, and configure theoptical cell to adjust a modulation of other light when the other lightpreviously emitted from the corresponding laser later passes through theplurality of optical elements to apply the second modulation to generatean adjusted version of the partial tile.